Stereoscopic display

ABSTRACT

A direct interaction stereoscopic display system that produces an augmented or virtual reality environment. The system comprises one or more displays and one or more beam combiners to virtually project high-resolution flicker-free stereoscopic 3D imagery into a graphics volume in an open region. Viewpoint tracking is provided enabling motion parallax cues. A user interaction volume co-inhabits the graphics volume and a precise low-latency sensor allows users to directly interact with 3D virtual objects or interfaces without occluding the graphics. An adjustable support frame permits the 3D imagery to be readily positioned in situ with real environments for augmented reality applications. Individual display components may be adjusted to precisely align the 3D imagery with components of real environments for high-precision applications and also to match accommodation-vergence distances to prevent eye strain. The system&#39;s modular design and adjustability allows display panel pairs of various sizes and models to be installed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 USC §119 to U.S. ProvisionalPatent Application Ser. No. 61/954,543, filed Mar. 17, 2014, and titled“COMPACT DYNAMICALLY ADJUSTABLE IMMERSIVE STEREOSCOPIC DISPLAY ANDDIRECT INTERACTION SYSTEM,” the entire contents of which are herebyincorporated by reference.

BACKGROUND

1. Technical Field

The present disclosure relates generally to interactivethree-dimensional (“3D”) displays. More particularly, the presentdisclosure addresses apparatus, systems, and methods making up a displaysystem with 3D in situ visualization that can maintain the eye's naturalaccommodation-vergence relationship.

2. Description of Related Art

Interactive 3D display systems have been the subject of a number ofdevelopmental efforts over the past 30 years. The prospect of reachingout and directly interacting with virtual content is universallyintriguing and may allow for step changes in creativity and efficiencyin developing models, creating art, and understanding and manipulatingcomplex data. Several groups have pursued merging 3D interactivedisplays in-situ with real environments. One aim of these groups hasbeen to enable real-time guidance for critical tasks where there islimited visibility. Another aim has been to allow for accurate andintuitive in-field visualization of complex data.

Medicine is one of the fields that stand to benefit the most fromdirectly interactive 3D display systems with in situ visualization.Surgeons are required to carry out operations in the least amount oftime and with minimal invasiveness. Understanding the layout of thepatient's internal anatomy may allow surgeons to plan the shortest andmost direct path for completing operations. While CT, MRI, andultrasound scans accurately lay out a patient's anatomical information,during surgery these modalities are usually displayed on monitors out ofthe field of view of the patient's body. The result is that surgeonshave to mentally store scanned patient data from one view and transformand apply it to the view with the patient. A few methods have beendeveloped to provide co-location of scanned data with the patient.

Head-mounted stereoscopic displays (“HMD”) were proposed in some effortsas a solution, but these are heavy and awkward to use because the cablesrunning to the HMD can restrict the freedom of movement of the user.HMDs are limited to displaying content at a single fixed or finite setof focal lengths. The focal length for single focal length HMDs isusually set at infinity while patient images from the display's stereoscreen converge at the actual distance of the patient (usually arm'slength or less). This disparity may result in accommodation-vergenceconflict where the eyes converge on a plane at a certain distance butare accommodated at a plane at another distance. Breaking of the naturalaccommodation-vergence relationship can lead to eye fatigue and resultin difficulty achieving optical fusion where left and right images nolonger appear fused. One HMD has been designed with three focal lengths.In this system, software toggles between the three fixed focal lengthsand infers the closest appropriate length based on the position of theuser in relation to the virtual content. This solution could bring thedisparity in accommodation-vergence closer in line. However, as thenature of surgery requires surgeons to arbitrarily move closer topatients for more detail and further away to establish the overalllayout, there would be frequent significant disparities in the regionsbetween the focal lengths.

Head-mounted displays are also especially prone to temporal misalignmentin the imagery as a result of latency. This latency is significantduring fast head movements, and in augmented reality applications themagnitude of the latency is intensified in proportion to the distancebetween the viewer and the subject. In medical settings, the distancebetween the surgeon and patient can be enough to introduce significantlatency issues. Another issue with using head-mounted displays insurgical settings is that assistants are not able to observe with thesurgeon the augmented graphics presented in context with the patientunless they themselves are wearing head-mounted displays, which addsadditional cost and complexity to the system. Assistants are usuallyleft to follow along on standard overhead displays, with the originaldisadvantage of not being able to fuse the patient data with actualanatomy.

Various additional display systems implemented to provide interactive 3Ddisplay systems with in situ visualization may use combinations oftechnique and/or equipment such as projection of images usingsemi-transparent mirrors, sensors to track the viewer's head to overlaya virtual view co-located with a subject, and stereoscopic viewingdevices. Such display systems exhibit several shortcomings. For example,such display systems may result in the viewer repeatedly shifting focusbetween the projected image plane and the subject, which is unintuitiveand could lead to, for example, blurred vision and/or inaccuratemovements during surgery. Other shortcomings of such display systems mayinclude large system footprints, reduced access to patients, weak andexpensive equipment, latency in viewer movement tracking, and inducementof eye strain, fatigue, and dizziness in the viewer.

SUMMARY

In one embodiment, a display system is disclosed. The display systemincludes a user interaction volume, a first display for displaying afirst image, a second display for displaying a second image, a firstbeam combiner, a second beam combiner, one or more tracking sensors, anda processor.

The first beam combiner is positioned at least partway between the firstdisplay and the second display and is configured to receive, and tooptically overlay, the first and second images. Each of the firstdisplay and the second display is concurrently devoted to either theleft or the right stereo image channel. The first beam combiner has asubstrate surface at least partially facing one display, wherein lightfrom that display is transmitted towards the user interaction volume.The first beam combiner also has a mirrored surface at least partiallyfacing the second display, whereat light from the second display isreflected towards the user interaction volume.

The second beam combiner is offset from first beam combiner in thedirection of the user interaction volume. The second beam combiner isconfigured to receive and to optically overlay the combined first andsecond images relayed from the first beam combiner with a view of theuser interaction volume. The combined two images form respectivestereoscopic left eye and right eye images of a virtual environment,whereby a user looking at the second beam combiner from a user viewposition perceives the virtual environment reflected from the secondbeam combiner as originating from within the user interaction volume.

The second beam combiner has a substrate surface at least partiallyfacing the user interaction volume, wherein light from the userinteraction volume is transmitted towards the user view position. Thesecond beam combiner also has a mirrored surface at least partiallyfacing the first beam combiner, wherein light from the first beamcombiner is reflected towards the user view position

The tracking sensors are arranged to sense at least a first input withinthe user interaction volume and at least a second input within a secondvolume region The first input includes position and orientationinformation of at least one user-controlled object. The second inputincludes user viewpoint position and orientation information. Theprocessor is arranged to receive the second input and adapt positioningof the images of the virtual environment so that the virtual environmentappears visually aligned with the user interaction volume according to aperspective of the user. The processor is also arranged to receive theposition and orientation information of the at least one user-controlledobject and determine a corresponding position and orientation in thevirtual environment. The processor is also arranged to update thevirtual environment based on the corresponding position and orientation.

The present disclosure will now be described more fully with referenceto the accompanying drawings, which are intended to be read inconjunction with both this summary, the detailed description, and anypreferred or particular embodiments specifically discussed or otherwisedisclosed. This disclosure may, however, be embodied in many differentforms and should not be construed as limited to the embodiments setforth herein; rather, these embodiments are provided by way ofillustration only so that this disclosure will be thorough, and fullyconvey the full scope of the disclosure to those skilled in the art.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present disclosureare described with reference to the following figures, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified.

FIG. 1 is a perspective schematic view of an embodiment of an immersive,direct interaction stereoscopic display system according to variousembodiments of the present disclosure showing the locations of a virtualdisplay plane and virtual 3D work pieces;

FIG. 2 is a side schematic view of an embodiment of an immersive, directinteraction stereoscopic display system showing the location of avirtual display plane;

FIG. 3A is a side schematic view of an embodiment of an immersive,direct interaction stereoscopic display system showing fields of view ofhead tracking and display-tracking sensors;

FIG. 3B is a perspective schematic detail view of two types of displaytracking markers including one set that mounts temporarily on the screensurface for initial registration of the screen location and a separatemarker mounted on the side of the display panel for subsequent tracking;

FIG. 3C is a side schematic detail view of 3D polarized glasses showingattached fiducial markers;

FIG. 4 is a side schematic view of an embodiment of an immersive, directinteraction stereoscopic display system showing the field of view of asensor for tracking hands or user-controlled objects;

FIG. 5 is the perspective view of the user of an embodiment of animmersive, direct interaction stereoscopic display system showing amethod for interacting with virtual workpieces;

FIGS. 6A, 6B, and 6C are a series of views from the perspective of theuser of the system, which show that, as the user moves about thedisplay, the displayed virtual imagery is updated to appear from theproper perspective.

FIG. 7 is a side schematic view of an embodiment of an immersive, directinteraction stereoscopic display system showing a line of sight from thedisplay to the user with an exaggerated indication of correctabledivergence or misalignment between display images due to refraction oflight through the exaggerated thickness of a beam combiner substrate;

FIG. 8 is a side schematic view of an embodiment of an immersive, directinteraction stereoscopic display system showing components and anexample orientation of the components for conveying polarized images toa user wearing polarized glasses with polarizing lenses at the anglesshown;

FIG. 9 is a side schematic view of an embodiment of an immersive, directinteraction stereoscopic display system showing components and exampleorientation of the components for conveying polarized images from thedisplays to a user wearing polarized glasses with polarizing lenses atthe angle shown;

FIGS. 10A and 10B are side schematic views of an embodiment of animmersive, direct interaction stereoscopic display system shown withdisplay panels of various sizes installed, demonstrating a range ofadjustments available with the system;

FIGS. 11A, 11B, and 11C are side schematic views of an embodiment of animmersive, direct interaction stereoscopic display system including anadjustable cover used to prevent glare and block views to the lowerdisplay panel;

FIG. 12 is a front perspective view of an embodiment of an immersive,direct interaction stereoscopic display system showing additionaloptional components including a stylus-type input device as well as atracking marker for desktop or other working surfaces for 2D design oruser-input capturing applications;

FIGS. 13A and 13B are user perspective and front perspective schematicviews, respectively, of an embodiment of an immersive, directinteraction stereoscopic display system showing an additional optionalstylus-type input device used in an example 3D virtual sculptingapplication;

FIGS. 14A and 14B are front perspective and user perspective schematicviews, respectively, of an embodiment of an immersive, directinteraction stereoscopic display system illustrating the ability of thesystem to virtually project 3D patient internal anatomy data in situwith patient's bodies to assist in training, intraoperativevisualization, surgery rehearsals, and surgery planning;

FIG. 15 is a block diagram of software and hardware architecture for anembodiment of an immersive, direct interaction stereoscopic displaysystem;

FIG. 16 is a side schematic view of an alternative embodiment immersivedirect interaction stereoscopic display, which uses a passive-polarizedstereoscopic dual stacked panel display; and

FIG. 17 is a side schematic view of an alternative embodiment immersivedirect interaction stereoscopic display, which uses a spatial lightmodulator-based digital holographic display.

Corresponding reference characters indicate corresponding componentsthroughout the several views of the drawings. Skilled artisans willappreciate that elements in the figures are illustrated for simplicityand clarity and have not necessarily been drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help to improve understanding of variousembodiments of the present disclosure. Also, common but well-understoodelements that are useful or necessary in a commercially feasibleembodiment are often not depicted in order to facilitate a lessobstructed view of these various embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following description, reference is made to exemplary embodimentsin which the disclosure may be practiced. These embodiments aredescribed in sufficient detail to enable those skilled in the art topractice the concepts disclosed herein, and it is to be understood thatmodifications to the various disclosed embodiments may be made, andother embodiments may be utilized, without departing from the spirit andscope of the present disclosure. The following detailed description is,therefore, not to be taken in a limiting sense.

Reference throughout this specification to “one embodiment,” “anembodiment,” “one example,” or “an example” means that a particularfeature, structure, or characteristic described in connection with theembodiment or example is included in at least one embodiment of thepresent disclosure. Thus, appearances of the phrases “in oneembodiment,” “in an embodiment,” “one example,” or “an example” invarious places throughout this specification are not necessarily allreferring to the same embodiment or example. Furthermore, the particularfeatures, structures, or characteristics may be combined in any suitablecombinations and/or sub-combinations in one or more embodiments orexamples.

Systems described herein may be useful for a variety of applications.Scientific visualization, medical training and surgical intraoperativevisualization are some of the areas that stand to benefit fromembodiments of the present disclosure. The present system uses novelmeans to serve the needs of these industrial fields as well asconsumer-level applications and achieve high standards in the areaswhere previous systems have fallen short.

An embodiment of a compact, adjustable, direct interaction stereoscopicdisplay system is illustrated in FIG. 1. This figure illustrates onemethod of interaction by a user with an embodiment of the system. Aprocessor or processors, not pictured, manages all tasks associated withgraphics generation, tracking, interaction simulation, and all otherrelevant input and output operations. The user is presented with astereoscopic three-dimensional model 115 or interface that appears tooriginate from beneath the system in a user interaction volume in frontof the user. The term “user interaction volume” refers to a region inwhich a user may use their hands or other user-controlled objects tointeract with virtual objects or interfaces that appear co-located inthe same space. The display system virtually projects 3D imagery intothe interaction volume where the user utilizes stereoscopic cues and/oraccommodation cues to spatially locate the components of the 3D imagery.This further triggers the proprioceptive sense of the user, which is thesense of the position of one's joints and limbs. The system complieswith the subsequent intuitive impulse of the user to reach out todirectly “touch” or interact with the graphics. The system detects thelocations of the fingers or hands of the user or objects under thecontrol of the user and creates a co-located virtual representation ofthe hands or objects. The system simulates reactions between the virtualrepresentation of the hands or objects and the 3D virtual objects orinterfaces, which results in the user feeling as though he or she isdirectly interacting with the virtual environment. In other words, inthe embodiment depicted, hardware and software components of the systemfunction together to allow the user to interact with virtual models 115or interfaces directly using one or more of the hands of the user or oneor more user-controlled objects. To the user, it would seem as if the heor she were interacting with real physical objects or interfaces.“Direct interaction” is the phrase used herein to refer to the conceptof a user directly interacting with or manipulating virtual objects orinterfaces in a volume of space where the virtually projected 3D virtualobjects or interfaces and the physical interaction region areco-located. As the user moves about the display system, the virtualmodel 115 or scene is updated to appear in the correct perspective tofurther reinforce the illusion that the object or scene is actuallypresent.

The two display panels 101 and 102 present left and right channels ofstereoscopic images, which are relayed through a first and second beamcombiner 103 and 104, respectively. The combined image planes of thedisplays 101 and 102 fold into a single virtual image plane 114. Atleast one of the tracking sensors 108 b or 108 a, one of which 108 a isobscured in this figure, tracks the user viewpoint position based onmarkers attached to stereoscopic glasses 116, worn by the user. Here,the term “user viewpoint position” refers to the position andorientation of one or both of the eyes of the user. The systemcalculates the user viewpoint position in relation to the displaysystem. In one embodiment, an additional sensor 106, not shown in thisview, tracks the position and orientation of the display 102 in relationto the second beam combiner 104, by tracking the marker 107 attached tothe display 102. This information is then used to determine the positionand orientation of the virtual screen plane 114 relative to the system.

Using the information provided by one or both of the user viewpointtracking sensors 108 a and 108 b and the display component positiontracking sensor 106, the system calculates where to display the imagesof virtual objects 115 or interfaces so that they appear in the properperspective as the user moves about. This would allow a user to moveabout and see different sides of a virtual object to more intuitivelyunderstand its structure, for instance. In an embodiment, an additionallow-latency tracking sensor 105, especially suited for hand orsimilar-scale object tracking, captures the position and orientation ofthe hands and fingers of the user or other objects under control of theuser. As alluded to before, the system calculates the proximity of thehands or user-controlled objects to the virtual objects 115 orinterfaces and, based on rules generated by software, manipulates thevirtual objects or interfaces accordingly.

Continuing on to the means of supporting and adjusting the position ofthe display system, an embodiment of the system features an adjustablesupport arm 111 a with a range of adjustments including height, forwardand backward position, left and right position, horizontal swivel andvertical tilt to suit the ergonomic or functional needs of the user. Inone embodiment, the support arm 111 a attaches to a firmly mountedvertical pole 111 b. The mount that attaches to the pole 111 b may beeasily loosened and re-tightened to a range of heights to allow, forinstance, for primarily sit-down or stand-up use of the system. Inanother embodiment of the system, the support arm features a high rangeof motion allowing for sit-down or stand-up use so that the pole mountmay remain fixed in one position without the need to readjust. Theadjustable support arm 111 a and compact design of the systemadvantageously allows the system to be positioned in close proximity toreal world environments for various mixed reality applications. Some ofthese applications are discussed in more detail in later sections.

A pair of display panels 101 and 102 are attached to the frame withquick-install type mounts 118 in a embodiment of the present disclosure.The quick-install mounts 118 use standard VESA hole patterns to allow avariety of display panel pairs to be used with the system. The framework113 that supports the displays 101 and 102 has multiple mounting holesor slots where the quick-install mounts 118 attach to allow theinstallation of display pairs of a variety of sizes. The displays panels101 and 102 used in the system should ideally be matching models withsimilar screen size and polarization. In the embodiment shown, thesystem will accommodate a variety of display pair screen sizes, forinstance, from less than 23″ up to slightly over 27″ LCDs. In theembodiment shown, the framework 113 supports the display screens 101 and102 at a 90-degree angle. Some embodiments of the system are envisionedto be equivalently serviceable with a frame that supports the displaysat an angle greater than or less than 90 degrees.

A first beam combiner 103 is attached to an adjustment apparatus 112that mounts to the display system support frame 113 in one embodiment.The adjustment apparatus 112 includes adjustments for tilt and elevationfor the left and right side so that the first beam combiner 103 may bepositioned substantially perfectly or perfectly at the bisecting anglebetween the display panels 101 and 102. The adjustable mount 112 alsoincludes a means to adjust the thrust distance of the first beamcombiner 103 along the bisecting angle between the first and seconddisplay panels 101 and 102. This thrust distance adjustment isincorporated to allow the use of display panels of various sizes. Thethrust adjustment also creates room for the user to have a full view ofthe virtual screen plane 114 when adjustments are made to the positionand orientation of the virtual screen plane 114.

Also shown in FIG. 1 is an optical filter 109 that incorporates ahalf-wave plate and a linear polarizing film. The optical filter 109 isused to rotate the natural polarization angle of the display 102 by 90degrees. This is necessary for when the system uses in-plane switching(IPS) display panels or displays panels which have a naturalpolarization angle that is perpendicular or parallel to the sides of thedisplay. This topic is covered in further detail in a subsequentsection.

Reference is now made to FIG. 2, which shows a side schematic view of anembodiment of the system. In this view, the virtual screen plane 114 isillustrated and sight lines from the user to the top and the bottomedges are shown. The virtual screen plane 114 can be adjusted closer toor further from the user and set at various angles by adjusting thesupport arm 110 of the second beam combiner 104. In various embodiments,the support arm 110 is adjustable to allow the second beam combiner 104to be set to a variety of positions and orientations set by the user. Insome embodiments, the second beam combiner 104 may be set into positionby simply grasping and moving it, without needing to loosen or tightenadjusting screws or use any tools. The ability to arbitrarily repositionthe virtual screen plane 114 also allows the accommodation distance ofthe graphics to be set to match the convergence distance of the eyes. Inother words, this means that the graphics will appear in focus at thesame plane at which the eyes are converging. This is the mostcomfortable way to view stereoscopic images. While people have sometolerance for accommodation-convergence mismatch, if the mismatch is toolarge, the image goes out of focus or stereo fusion is lost. Thisadjustment ability also permits the virtual image plane 114 to bepositioned in close proximity to or directly overlaying components inreal environments for augmented reality applications.

FIG. 2 also illustrates the distance of the virtual screen plane inrelation to the arm reach of a user. For direct interactionapplications, the image plane 114, at the adjusted distance shown, iswell within a comfortable reaching distance of the majority of potentialusers. Regarding the viewing angle, previous ergonomic studies haveshown that, a significantly downward gaze angle is the best angle forclose work. This can be verified, anecdotally, by considering how mostpeople tend to read books and magazines at a low downward angle asopposed to at horizon level. Ergonomic studies have, in fact,recommended positioning displays at up to a 50° angle below horizonlevel for displays at a 25″ distance and to an even greater downwardangle for closer viewing distances. The virtual image plane 114 in theembodiment depicted in FIG. 2 shows the virtual image plane 114 ataround a 19″ distance from the eyes of the user and at a downward angleof 55° below the horizon. In this position, the virtual screen plane 114is well within comfortable reach of the user and interaction withvirtual objects or interfaces presented by the display is facilitated.

FIGS. 3A to 3C show a side schematic view of an embodiment of thesystem, a perspective detail view of display tracking markers 107, and adetailed side view of the stereoscopic polarized glasses 116 worn by theuser for various embodiments of the system. FIG. 3A illustrates thefield of view cone 108 c of a user-viewpoint tracking sensor 108 a andthe field of view cone 106 a of a display component tracking sensor 106.The position and orientation of the display 102 is tracked by the sensor106 which captures images of the tracking marker 107, which is rigidlyattached to the display panel 102. The sensor 106 relays images to aprocessor, not displayed, where software-generated algorithms are usedto calculate the position and orientation of the screen 102 a of thedisplay panel 102. The calculation takes into account thepreviously-gathered information on the location of the screen 102 a inrelation to the position of the tracking marker 107. The location of thescreen 102 a is required to calculate the virtual screen plane 114location. This location information is then used to facilitate thesimulation of direct interaction with virtual objects 115 or interfaces.A registration routine is used upon system setup to establishinformation about the position and orientation of the screen 102 a inrelation to the tracking marker 107.

FIG. 3B illustrates the components used for initially registering theposition and orientation of the screen 102 a in relation to the displaytracking marker 107 in some embodiments. When first setting up thesystem, or when a new display panel pair is installed on the system, thetracking marker 107 is attached on the lower side of the display panel102 near the center of the field of view 106 a of the display componenttracking sensor 106. To register the precise location of the screen 102a of the display panel 102, removable tracking markers 117 a and 117 bare attached on the face of the screen 102 a. In the embodiment shown inFIG. 3B, the tracking markers 117 a and 117 b feature a square corner.In this embodiment, one of the tracking markers 117 a is temporarilymounted on the face of the screen 102 a so that the square corner of thetracking marker 117 a aligns with the lower right corner of the screen102 a where the pixels terminate. The other tracking marker 117 btemporarily mounts at a given percentage horizontal and verticaldistance away from the lower right screen corner, which may be indicatedby a cross hair displayed by the system at the correct location. Uponinitial setup of the system, or whenever a new display panel pair isinstalled with the system, the sensor 106 captures images of all threetracking markers 107, 117 a and 117 b, after all have been attachedaccording to the guidelines previously outlined. The system registersthe location of the display screen itself 102 a in relation to thetracking marker 107. After the initial registration, the temporarytracking markers, 117 a and 117 b are removed and the location of thedisplay screen 102 a is thereafter determined using solely the trackingmarker 107 mounted on the side of the display panel 102. In someembodiments, the initial registration of the position and orientation ofthe screen 102 a may also be accomplished using a hand-held, tracked,pointed input device which may be pointed at previously specified pointson the screen. In other embodiments, the position and orientation of thescreen 102 a may be calculated by using the marker 107 attached at aspecific position on the panel in conjunction with stored geometricinformation about the display panel 102.

Returning to FIG. 3A, the user viewpoint position tracking sensor 108 amay be one of two sensors mounted on the second beam combiner 104. Thissensor or sensors is configured to capture images within a field of view108 c, as shown. In some embodiments, the positions and orientations ofboth eyes are considered. The system determines the locations of theeyes, first, by capturing images of the tracking markers 116 a on theglasses 116 illustrated in FIG. 3C. Then, as the geometry of the glasses116 is known and stored in memory, the system calculates the view centerof the eyes. The system may then use stored information about averagehuman inter-ocular distances or, alternatively, reference an actualinter-ocular distance input by the user from some previous measurementto calculate the specific position and orientation of each eye. A singleuser-viewpoint position tracking sensor 108 a or 108 b is capable ofdetermining the eye locations of the user independently. However, insome embodiments, an additional sensor is used on the opposite side ofthe second beam combiner 104 to provide robust tracking for variousextreme instances. These instances include when the user moves closeenough to the sensor 108 a that the tracking markers 116 a move outsideof the field of view cone 108 c of the sensor 108 a, when the userrotates their head enough that the line of sight from one sensor 108 ato the markers 116 a is obstructed, or when extraneous objects migratein to the field of view and occlude the line of sight to the trackingmarkers 116 a.

Turning to FIG. 4, this side schematic view of one embodimentillustrates the field of view 105 a of the user interactionvolume-tracking sensor 105. In the embodiment depicted, this sensor is acommercially available, low-latency sensor designed to track one or moreobjects within a volume similar to the cone-shaped volume 105 aillustrated. The outline of the field of view 105 a of the sensor 105represents a portion of the range where the sensitivity of the sensor105 is highest. The sensor 105 has an even greater range with ideallighting conditions. The sensor 105 captures pose and position of thehand and fingertips as well as the tips and 3D vector directions to thetips of user-controlled objects such as pens, styli, pointers et ceterawith accuracy in the sub-millimeter range. Software linked to the sensor105 reconstructs virtual hands or virtual objects that closely matchesthe actual hand gestures or object movements with very low latency. Thecombination of low-latency, sub-millimeter accurate tracking combinedwith high resolution 3D imagery that co-locates with the userinteraction volume provided by the system opens the door forapplications that are extremely intuitive, immersive, precise andrealistic.

FIG. 5 shows a perspective view from the point of view of the user of asystem according to embodiments of the present disclosure, illustratingthe capability to simulate direct interactions between the user andvirtual objects. In the embodiment depicted, the system derives thelocations and poses of the hands of the user, then calculates theproximity of each finger or relevant portion of the hand to the nearestpart of the virtual objects. Software then manipulates the virtualobjects according to various rules that govern the reaction of virtualobjects to user inputs. FIG. 5 further illustrates the occlusion ofvirtual objects by the hands of the user. In this embodiment, the systemrecognizes the fingers, or parts of the hand that are nearer to the userviewpoint position and selectively generates only the portion of thegraphics that are not “behind” the hands. This further adds to theillusion that the hands or fingers or the user-controlled objects aredirectly interacting with the virtual objects or interfaces.

FIGS. 6A to 6C show perspective views from the viewpoint of a usermoving from left to right in front of the display system. These figuresillustrate the simulation of motion parallax effect by an embodimentsystem. This effect is achieved by tracking the display components aswell as the viewpoint of the user and by continuously calculating thecorrect corresponding view to display for each new user viewpointlocation.

Reference is now made to FIG. 7, which shows a side schematic view of anembodiment system illustrating the line of sight from the displays 101and 102 to the viewpoint position of the user. In the embodimentdepicted, the first beam combiner 103 and the second beam combiner 104are partially transparent mirrors with mirrored surfaces 131 and 141,respectively, in the orientations shown. The mirrored surface 131 of thefirst beam combiner 103 faces the second display 102. In thisorientation, the light from the second display 102 is directed to theuser-viewpoint position behind the glasses 116 without any secondary orghost images or refraction-related misalignment. The light from thefirst display 101 is transmitted through the substrate thickness of thefirst beam combiner 103, where it undergoes a slight shift 256 a and 256b due to refraction as it travels through the thickness of thesubstrate. The light then reflects off the mirrored surface 141 of thesecond beam combiner 104 towards the user viewpoint position. In theembodiment shown, the first beam combiner 103 is a partially transparentmirror, which is shown with exaggerated thickness in order todifferentiate the mirrored 131 from the substrate surface. The degree ofimage misalignment 256 due to refraction of the light from the firstdisplay 101 through the substrate thickness is also exaggerated. Inactuality, the thickness of the partially transparent mirror 103 is muchless and the image misalignment 256 due to refraction is negligible, orcan be effectively adjusted away when initially installing and settingthe position of the first display panel 101.

The view portrayed in FIG. 7 also illustrates how the partialtransparency of the second beam combiner permits the user tosimultaneously view his or her hand 250 with the virtual imagery. In theembodiment shown, the second beam combiner 104 is also a partiallytransparent mirror. The partially transparent mirrors used as the firstand second beam combiners 103 and 104 may have a variety of transmissionto reflection ratios. For instance, in some embodiments, both may beessentially partially transparent mirrors with a 50 to 50 transmissionto reflection ratio, or both may have unequal transmission to reflectionratios, such as 70 to 30. Other embodiments may include variousdifferent combinations of partially transparent mirrors with equal andunequal transmission to reflection ratios.

Turning now to FIGS. 8 and 9, these figures show side schematic views ofembodiment systems using different common display panel technologies.These views, specifically, illustrate the methods of deliveringpassive-polarized stereoscopic images with two common types of LCDs. Inthe embodiment systems described herein, the method of relayingstereoscopic images to the user is by passive polarization. Thoseskilled in the art will understand the principles of operation of linearpassive-polarized displays. In short, the left and right image channelsof a stereoscopic image are polarized at angles that are 90 degreesapart from each other. These images are selectively passed on to theleft and right eyes of the user through glasses 116 with polarizingfilters on the right and left lenses that are also set at 90 degrees toeach other. The left lens of the glasses use a polarizing filter thatcorresponds to the polarization angle of the display panel dedicated tothe left image channel and vice versa for the right image channel.

In another example, the display may be configured similar to setup shownin FIG. 8. However, instead of using a single half-wave plate affixed toone display to achieve stereo images via orthogonal linear polarization,a circular-polarization approach may be employed. In this case, aquarter-wave plate would be affixed to each display, and the user wearscircular polarized 3D glasses. The quarter-wave plates would circularlypolarize the light from each display 101, 102. The light from the firstdisplay 101 would reflect once off the second beam combiner 104, whichwould change the direction of the circular polarized light fromleft-handed to right-handed or vice versa. The light from the seconddisplay 102 would reflect twice: once off the first beam combiner 103,and once off the second beam combiner 104. This double reflection wouldreverse the handedness of the light polarization from the display twice,resulting in the light returning to its original handedness. The lightfrom each display 101, 102 would thus be opposite handedness from eachother, which would allow the user wearing circular polarized 3D glassesto perceive the stereo imagery.

Most LCDs have a natural linear polarization angle. For some displays,like twisted nematic or TN LCDs, as shown in FIG. 9, this angle lies ata 45-degree angle 180 and 181 to the sides of the display panel. For TNdisplays 101′ and 102′, when the display image from the lower verticaldisplay 102′ is reflected off of the first beam combiner 103, thepolarization angle 181 is flipped from 135 to 45 degrees 181 a. When theimages from both displays 101′ and 102′ reflect again off the secondbeam combiner 104 towards the user, the polarization angles of bothimages flip 90 degrees again, though they remain 90 degrees apart fromeach other. To receive the corresponding left and right-eye images usingthese TN-type display panels, the user simply wears stereoscopicpolarizing glasses 116 with the polarization angles of the left lens setat 135 degrees 116 e and the polarization of the right lens set at 45degrees 116 d or vice versa.

For other LCD panels, for instance in-plane switching (IPS) or verticalalignment (VA) displays, the natural polarization angle of the displayis parallel to or perpendicular to the sides of the display panel. IPSor VA-type LCDs are often preferable due to their better color qualitiesand wider viewing angles. However, when using displays with vertical orhorizontal polarization, as shown in FIG. 8 the polarization angle ofthe light 281 a from the display panel 102 in the lower position doesnot flip 90 degrees when reflecting off the mirror. Instead, it remainsat the same angle. Thus, the images from the displays 101 and 102 willarrive to the user with the same polarization angle and no separation ofleft and right stereo image channels will be possible. For this reason,in various embodiment incorporating IPS display panels, a simplecombination half-wave plate and linear polarizing filter 109 is attachedto the front of the lower display panel 102. The half-wave plate 190rotates the polarization angle of the lower display panel light 281 a by90 degrees and then the linear polarization filter 191 filters out anyportion of the light 281 b not rotated to the desired angle, with theresult being clean vertically polarized light 281 c. With this, theimage of the lower display 102 is rotated 90 degrees relative to theupper display panel 101. The left and right channels of light are thusable to be selectively delivered to the left and right eyes through the0 and 90-degree polarized lenses 116 b and 116 c of the glasses 116.

FIGS. 10A and 10B are side schematic views of an embodiment system,showing two sets of display panels of different sizes installed. FIG.10A shows the display system with a pair of 23-inch display panels 101and 102, while FIG. 10B the system with a pair of 27-inch display panels101″ and 102″. In the embodiment depicted, the frame 113, includesmultiple mounting holes or slots to suit a variety of display sizes fromunder 23″ to just over 27″. These figures further demonstrate how theadjustment ability of the support arm 110 of the second beam combiner104 and the thrust distance adjustment ability of the first beamcombiner 103 help to facilitate use of display panels of a variety ofsizes.

FIGS. 11A to 11C are side views of an embodiment system that include theuse of a glare-reducing cover 295 that adjusts to accommodate displaypanel pairs of various sizes as well as changes in adjustments to theinner components of the system. FIGS. 11A and 11B show the system withdisplay panel pairs of two different sizes. FIG. 11C shows the systemwith the same display panel pair as that used in FIG. 11B but with theinner components of the system adjusted so that the virtual image plane114, not shown, is at a farther focal distance.

FIG. 12 depicts a possible setup for design applications. A stylus 260with attached tracking markers is used to write, draw, or otherwisedesign on a working surface 264. The tracking sensor 105 captures theposition and orientation of the stylus 260 along with the position andorientation of a tracking marker 262 set on the working surface 264. Thesystem uses previously established data on the geometry of the stylus260 to calculate the location of the stylus point 260 a and the vectordirection of the stylus body. The system then determines when the styluspoint 260 a makes contact with the working surface 264 and relays thestroke marks made by the user to application software. Thisstroke-capturing capability may be useful for a variety of tasksincluding 2D art and design, handwriting capturing, digitization ofexisting drawings, blueprints, or patterns, and true-to-scale capturingof profiles of flat-shaped parts, tools et cetera.

A virtual sculpting application is depicted in FIGS. 13A and 13B. Theseuser perspective and front views depict one embodiment system with theinclusion of a stylus 260 that may be useful for various design andcoordinate measuring applications. The position and orientation of thestylus 260 is captured by the sensor 105. The 3D position information ofthe stylus 260 is relayed to a processor where it may be used as inputfor a variety of design and measurement software applications. Virtualsculpting and 3D coordinate measurement are two examples of applicationswhere the use of a tracked stylus 260 may be advantageous. Given therobust tracking capability of the sensor 105, the fingers or hands ofthe user may certainly be used as inputs for virtual sculpting or 3Dcoordinate measuring applications. However, in some instances, users mayfeel a greater sense of precision or simply be more accustomed to usinga pointed device such as the stylus 260.

In FIGS. 14A and 14B, an augmented reality application for medical useis depicted. These figures show front and user viewpoint perspectiveviews of an embodiment system set up with the components to track apatient 266 and overlay a virtual model of the internal anatomy 267 of apatient onto the actual view of the patient 266. In these figures, thesystem is arranged so that the images of the internal anatomy 267 areupdated to appear to the user in the correct perspective from whereverthe user is positioned in front of the display system. Fiducial markers268, represented only schematically in these figures, are attached tothe patient or alternatively on to components fixed rigidly to thepatient. The markers 268 should be compatible to whichever imagingmodality is used for scanning the internal anatomy of the patient. Themedical scanning system stores the internal anatomy geometry data of thepatient 266 in relation to the location data of the tracking markers 268and relays all of this information to a processor, not shown in thefigures. The processor compares the data regarding the tracking markers268 from the medical imaging source with the live data feed from thesensor 105 and uses algorithms to determine the correct degrees totranslate, rotate and scale the anatomy imagery 267 in three dimensionsin order for the internal anatomy to appear to coincide with the patient266. This ability to overlay high quality medical imagery in situ withpatients in real time may be highly beneficial for a variety of medicalscenarios including training, surgery rehearsals, surgery planning andintra-operative visualization.

A block diagram of software and hardware architecture for one embodimentinteractive display system is shown in FIG. 15. This figure illustratesthe essential hardware and software components used by the embodimentdisplay system to receive user inputs and translate these inputs intocommands that influence or change virtual environments or interfaces.The various tracking sensors 316, 314, and 312 relay data to a processoror multiple processors 302 working in parallel. One sensor 316 tracksthe viewpoint position of the user while another 314 tracks the positionof user-controlled objects, including possibly the hands of the user orhand-held input devices such as a stylus. The user-controlled objectsensor 316 may also track environment markers such as a 2D writing planemarker 262, which, for instance, may be positioned on a desktop 264 asseen in FIG. 12. An additional sensor 312 tracks the position andorientation of the display system components in relation to each other;for instance, it tracks the position and orientation of the second beamcombiner 104, not shown in this figure, in relation to the rest of thedisplay system.

Tracking sensor modules 311, 313, and 315 interpret and translate thetracking data from the tracking sensors 312, 314, and 316 into formatsthat are usable by a virtual alignment module 317. In the embodimentdepicted, all tracking sensors 312, 314, and 316 are mounted on thesecond beam combiner 104 at established positions. The virtualenvironment alignment module 317 receives the user viewpoint,user-controlled object and the display component position informationand determines the locations of the virtual screen plane 114, the userviewpoint position and the position of the hands 250 of the user oruser-controlled objects 260 in relation to the second beam combiner 104.When virtual object or scenery data 305 is called up by applicationsoftware 305, the virtual environment alignment module 317 determinesthe correct perspective to display the 3D images of the virtual objectsor scenery. The virtual environment alignment module 317 establishes ageometric framework for this purpose, which is built off of the knowngeometric relations between the tracking sensors and the second beamcombiner 104. This geometric framework essentially determines where toplace and point a set of “virtual cameras” in order to captureperspective-correct stereoscopic views of the virtual objects orscenery. The virtual environment alignment module 317 then instructs theprocessor or processors 302 to relay images of the virtual objects orscenery in the correct perspective views to displays 306 in the displaysystem, which helps to ultimately recreate the 3D image of the virtualobjects or scenery to the user in the appropriate perspective. Becausethe viewpoint position of the user is tracked, the virtual environmentalignment module 317 is able to update the virtual object or scenerygraphics so that they appear to the user to be spatially fixed in placeat the correct perspective even as the user moves about in front of thedisplay system. The virtual environment alignment module 317 alsoestablishes a geometric framework pertaining to the location ofuser-controlled objects including, for instance, the hands of the useror a stylus, in relation to the virtual screen plane so that applicationsoftware 305 may use the locations and movements of one or more fingersor hands or one or more user-controlled objects as inputs. The inputsare utilized by the application software 305 to interact with ormanipulate virtual objects, scenery or other interfaces according torules written in the application software 305.

FIGS. 16 and 17 illustrate alternative embodiment direct interactiondisplay system designs using two different types of high-resolutiondisplays. The use of only one display panel allows the bulk of thesystem to be minimized for applications where space is at a premium. Thetradeoff for smaller system size is a somewhat higher cost for either ofthe display types, though there would be only one display panel used forsuch a system.

The display 401 shown in FIG. 16 is a dual stacked panelpassive-polarized stereoscopic display. This type of display uses twostacked LCD screens inside of one display housing and passivepolarization to deliver a full-native resolution stereoscopic picture.The display produces full-resolution left and right stereo images, eachwith separate polarization angles. The light from the display 401reflects off the mirrored surface 430 of a partially transparent mirror403 and passes through a pair of passive-polarized glasses 416 withlenses arranged to selectively accept the corresponding left orright-screen images.

The display representation 411 shown in FIG. 17 is a spatial lightmodulator-based digital holographic display, developed by SeeRealTechnologies of Luxembourg. With this display design, there are norequirements for any type of glasses to see virtual objects or scenes in3D. One or more sensors, not shown, adjacent to the spatial lightmodulator display 411, track the eyes 426 of the user. A processor, notshown, connected to the display 411 receives the information from thetracking sensors and performs holographic synthesis calculations for thevirtual objects or scenes and determines a complex wave-field for theobjects or scenes to be represented by the spatial light modulator-baseddisplay. The display creates a 3D wave-field holographic image that isvisible only within a viewing zone, called a “viewing window” 428, atthe location of the eyes 426 of the user. The display can produce imageswith true depth of focus that corresponds with the geometries of thevirtual 3D objects or scenes. Light from the display 411 reflects offthe mirrored surface 440 of a partially transparent mirror 413 towardsthe viewing window 428 directly adjacent to the eyes 426 of the user.The systems depicted in FIGS. 16 and 17 are again schematicallyrepresentations, and, though not shown in the figures, embodimentsystems may include a tracking sensor or sensors for the userinteraction volume below the partially transparent mirror as well as asensor or sensors for tracking the user-viewpoint.

Thus the reader will see that at least one embodiment of the directinteraction stereoscopic display system can provide a combination ofunique features including a full-resolution picture that is comfortableto view, a highly intuitive and responsive interaction interface, arobust software and hardware framework that uses low-cost components anduses system resources efficiently, and a flexible hardware design thataccommodates the ergonomic and functional needs of the user.

While the above description contains many specificities, these shouldnot be construed as limitations on the scope, but rather as anexemplification of several embodiments thereof. Many other variationsare possible. For example, the displays 101 and 102 in the two-displayvariation shown in FIG. 2, may be mounted in reverse from what is shownso that the rear side of the lower vertical monitor 102 faces the eyesof the user and the support arm 110 of second beam combiner 104 attachesonto an extended portion of the display system support frame 113.

In another example, the second beam combiner 104 may not attach to thesupport arm 110 as shown in FIG. 2, but instead may attach to one ormore planar linkage arms that connect to the left or right or both leftand right edges of the frames of the first and second beam combiners 103and 104, as seen from the perspective of the user.

In yet another example, the sensor or sensors 108 a and 108 b fortracking the user viewpoint, the sensor 106 for tracking the displaycomponent position, and the interaction volume tracking sensor 105 maynot all be mounted adjacent to the second beam combiner 104. All sensorsor any combination of the said tracking sensors may be made to functionequivalently being attached in various combinations adjacent to otherdisplay system components including the first beam combiner 103, thesecond display 102, the first display 101 or the display support frame113. The equivalent function of the sensors placed in various locationsis realized by mounting the sensors so that there is a clear view to thetracked regions. The display component tracking sensor 106 may be madeto, instead, track the location of the second beam combiner 104. In thiscase, the tracking marker 107 is not used. Instead, a tracking marker isattached to the second beam combiner within view of the displaycomponent-tracking sensor 106 wherever it is installed. In this andother envisioned examples, the system has access to the relevantlocations of all tracking sensors in relation to each other forwhichever orientation of sensors is chosen, with the result that theuser experience is ultimately identical from one orientation to another.

According to various embodiments of the present disclosure, the displaysystem may feature motion-sensing to allow for direct interaction andhead tracking for correct viewer-centered perspectives. The combinationof these features may make this system beneficial for many purposes,including but not limited to: spatially-aligned visualization, training,design and gaming. Thus the present disclosure may have application in avariety of fields, including medicine (e.g. preoperative surgeryplanning, rehearsals and intraoperative spatially aligned visualization,stereotactic surgery, telemedicine), medical education and training(e.g. neurology, dentistry, orthopedics, ophthalmology, et cetera),complex 3D data visualization and processing (e.g. biotechnology,computational chemistry, cartography, and geological engineering), thedesign arts (e.g. industrial design, 2D drawing, 3D model creation, 3Danimation), engineering applications (e.g. 3D modeling, virtualprototyping, assembly inspection, mixed reality test fitting, analysisresults processing), and entertainment (e.g. gaming).

Accordingly, there may be several advantages of one or more aspects ofthe direct interaction stereoscopic display system disclosed herein. Oneis that embodiments of the display system may offer a full-resolutionstereoscopic picture without flicker. Another advantage is thatembodiments of the display system may utilize one or more low-latencyand highly accurate motion sensors to capture user gestures or themovements of user-controlled objects, allowing for seamless directinteraction and enhancing the sense of immersion to users. Additionally,in one or more embodiments of the display system, the interactionvolume, and the visualization volume (i.e., the volume below the displaywhere 3D images of virtual objects or interfaces appear) are co-located.The advantage here is that as users interact directly with virtualobjects or interfaces presented in front of them, the proprioceptivesense (innate sense of the position of one's limbs or joints) of theuser may be utilized in addition to the stereoscopic vision cues,resulting in a high degree of realism and immersion. Another advantageis that embodiments of the present disclosure can effectively track theuser-viewpoint using only one sensor and effectively track the userinteraction area using only one sensor, which may lessen the CPU loadand allow the use of a less expensive computer.

Additional advantages of one or more embodiments of the display systemare that the overall system may be relatively lightweight and mounted ona mobile stand, which could allow the system to be used in a sitting orstanding position or any position between, and which may further allowthe system to be easily positioned to overlay 3D graphics in situ withreal environments to facilitate augmented reality applications. In oneor more embodiments, all tracking sensors are mounted directly oncomponents of the system. The resulting advantage may be a compactsystem footprint as well as greater freedom of movement about thedisplay without the risk of interfering with separately-mounted trackingsensors as is the case with previous approaches.

Other advantages of one or more aspects include the use of simple,lightweight and inexpensive passive-polarized glasses, which may providea smooth high-fidelity picture without flicker. Additionally, thedisplay system components may be dynamically adjusted in one or moreaspects providing multiple advantages including the ability to installdisplay panel pairs of a variety of sizes to suit the needs of the user.Another advantage of the dynamically adjustable display components inone or more aspects may be an ability to easily adjust the location ofthe virtual image plane, which is the image plane that the displayvirtually projects below the display system, so that the 3D imagery canbe precisely co-located with real environmental components to facilitatehigh-precision augmented reality applications. The ability to set thevirtual image plane location may also allow the user to keep theaccommodation and vergence distances for 3D virtual images in parity,which can reduce and/or minimize eyestrain for users. Another advantagestemming from the ability to dynamically adjust display components inone or more aspects is that the gaze angle of the user to the virtualimage plane may be easily adjusted to a variety of angles including moreergonomically-correct downward gaze angles, which are appropriate forup-close work.

Some additional advantages of one or more aspects relate to theutilization of a modular design for the display system, including theuse of quick-install display mounts with VESA-standard mounting holes toallow for easy installation of display panels of a variety of sizes andmodels. This advantage may allow users the freedom to choose a displaypair that precisely fits their needs and budget. An additional advantageof the modular design of the display system in one or more embodimentsis that the process to upgrade or service components of the system mayconsequently be simpler and more straightforward. Further advantages ofone or more aspects may be apparent from a consideration of the drawingsand ensuing description.

Although the present disclosure is described in terms of certainpreferred embodiments, other embodiments will be apparent to those ofordinary skill in the art, given the benefit of this disclosure,including embodiments that do not provide all of the benefits andfeatures set forth herein, which are also within the scope of thisdisclosure. It is to be understood that other embodiments may beutilized, without departing from the spirit and scope of the presentdisclosure.

What is claimed is:
 1. An interactive display system, comprising: a userinteraction volume; a first display for displaying a first image; asecond display for displaying a second image; a first beam combinerpositioned at least partway between the first display and the seconddisplay, the first beam combiner configured to receive, and to opticallyoverlay, the first and second images, whereby each of the first displayand the second display is concurrently devoted to either the left or theright stereo image channel, the first beam combiner comprising: asubstrate surface at least partially facing one display, wherein lightfrom that display is transmitted towards the user interaction volume; amirrored surface at least partially facing the second display, whereatlight from the second display is reflected towards the user interactionvolume; and. a second beam combiner offset from first beam combiner inthe direction of the user interaction volume, the second beam combinerconfigured to receive, and to optically overlay, the combined first andsecond images relayed from the first beam combiner with a view of theuser interaction volume, the combined two images forming respectivestereoscopic left eye and right eye images of a virtual environment,whereby a user looking at the second beam combiner from a user viewposition perceives the virtual environment reflected from the secondbeam combiner as originating from within the user interaction volume,the second beam combiner comprising: a substrate surface at leastpartially facing the user interaction volume, wherein light from theuser interaction volume is transmitted towards the user view position; amirrored surface at least partially facing the first beam combiner,wherein light from the first beam combiner is reflected towards the userview position; and one or more tracking sensors arranged to sense atleast a first input within the user interaction volume and at least asecond input within a second volume region, wherein the first inputincludes position and orientation information of at least oneuser-controlled object, and wherein the second input includes userviewpoint position and orientation information; and a processor arrangedto receive the second input and adapt positioning of the images of thevirtual environment so that the virtual environment appears visuallyaligned with the user interaction volume according to a perspective ofthe user, the processor further arranged to receive the position andorientation information of the at least one user-controlled object, anddetermine a corresponding position and orientation in the virtualenvironment and update the virtual environment based on thecorresponding position and orientation.
 2. The interactive displaysystem according to claim 1, further comprising a support for the firstdisplay and the second display and the first beam combiner and thesecond beam combiner, the support adapted to allow adjustments to thefirst display and the second display and the first beam combiner,whereby an image plane of the first display and an image plane of thesecond display may be brought into alignment which each other to theuser.
 3. The interactive display system according to claim 2, whereinthe support comprises a frame, the system further comprising a framesupport to position the frame above the user interaction volume, theframe support adapted to allow adjustments to a frame height, a frameforward position, a frame backward position, a frame left position, aframe right position, a frame horizontal swivel, and a frame verticaltilt, whereby the support may be adjusted to suit the ergonomicrequirements of the user.
 4. The interactive display system according toclaim 3, further comprising a means to carry out said adjustments to theframe through a single point of application whereby users can manipulatedisplay through the range of said adjustments using a single motion. 5.The interactive display system according to claim 2, wherein the supportcomprises a frame, the system further comprising one or more trackingsensors arranged to sense the second input and a third input, whereinthe third input includes position and orientation information of one ofthe first display and the second display, wherein the processor isadapted to receive the second input and the third input and calculate aviewable screen size, an image plane position, and an orientation of thefirst display or the second display relative to the display system tomake corrections to a virtual environment camera position in order forthe virtual environment to appear visually aligned with the userinteraction volume according to the user viewpoint.
 6. The interactivedisplay system according to claim 5, wherein the frame rigidly supportsat least the first display, the second display, and the first beamcombiner in a fixed spatial relationship, the system further comprisinga second support for the second beam combiner in such a way that theposition and orientation of the second beam combiner may be arbitrarilyadjusted relative to the frame, and wherein the one or more trackingsensors are mounted adjacent to the second beam combiner, wherein theprocessor is arranged calculate the position and orientation of thesecond beam combiner in relation to the display system and directadjustment of the virtual environment camera position in order for thevirtual environment to appear visually aligned with a user interactionregion, whereby the image plane of the second display may berepositioned to suit a service or an ergonomic requirement of the user.7. The interactive display system according to claim 6, wherein a secondbeam combiner height, a second beam combiner forward position, a secondbeam combiner backward position, and a second beam combiner verticaltilt may be adjusted relative to the frame.
 8. The interactive displaysystem according to claim 6, wherein a user viewpoint tracking sensor iscoupled to the second beam combiner, thereby centering a field of viewof the viewpoint tracking sensor on the user.
 9. The interactive displaysystem according to claim 6, further comprising one or more trackingsensors arranged to sense an object input, wherein the object inputincludes position and orientation information of at least one object,wherein the processor is further arranged to receive the object inputand determine a corresponding position and orientation in the virtualenvironment and use the object input to update the virtual environment,thereby causing the virtual environment to appear visually aligned tothe object according to the perspective of the user.
 10. The interactivedisplay system according to claim 6, further comprising one or moretracking sensors arranged to sense an object input, wherein the objectinput includes position and orientation information of at least oneobject, the object input having a field of view centered on theuser-controlled object.
 11. The interactive display system according toclaim 9, wherein only one set of sensors is adapted to track the one ormore user-controlled objects and the one or more objects in thereal-world environment.
 12. The interactive display system according toclaim 1, wherein the processor is adapted to receive eye calibrationdata indicating positions of a left eye and a right eye of the user withrespect to a position and an orientation of the user viewpoint andwherein the processor is adapted to generate a stereoscopic left imageand a stereoscopic right image based on the eye calibration data and thesecond input.
 13. The interactive display system according to claim 12,wherein the eye calibration data comprises a calculated user view centerand an inter-ocular distance to generate a distinct left eye positionvalue and a distinct right eye position value.
 14. The interactivedisplay system according to claim 1, wherein the user-controlled objectcomprises at least one of the hands of the user.
 15. The interactivedisplay system according to claim 1, wherein the user-controlled objectcomprises at least one stylus device.
 16. The interactive display systemaccording to claim 1, wherein the user-controlled object comprises atleast one haptic device.
 17. An interactive display system, comprising:a user interaction volume; a dual stacked panel passive-polarizedstereoscopic display for displaying stereoscopic images to a user orusers wearing corresponding passive-polarized glasses; a beam combinerpositioned at an acute angle from the display, the beam combinercomprising: a substrate surface at least partially facing the userinteraction volume, wherein light from the user interaction volume istransmitted towards the user view position; a mirrored surface at leastpartially facing the display, wherein light from the beam combiner isreflected towards the user view position; and; one or more trackingsensors arranged to sense at least a first input within the userinteraction volume and at least a second input within a second volumeregion, wherein the sensed first input includes position and orientationinformation of at least one user-controlled object, and wherein thesensed second input includes user viewpoint position and orientationinformation; and a processor arranged to accept the user viewpointposition and orientation information, and adapt the virtual environmentcamera position so that the virtual environment appears visually alignedwith the user interaction region according to the perspective of theuser, the processor further arranged to accept the user-controlledobject position and orientation information, and determine acorresponding position and orientation in the virtual environment anduse this information to update the virtual environment based on thecorresponding position and orientation.
 18. An method for displaying a3D image to a viewer at a view position, comprising: at a first display,generating a first image having a first polarization, the first imagecorresponding to a stereoscopic first view of a virtual environment; ata second display, generating a second image having a secondpolarization, the second image corresponding to a stereoscopic secondview of the virtual environment; at a first beam combiner positioned atan acute angle from the first display and the second display, passingthe first image through the first beam combiner toward a second beamcombiner and reflecting the second image toward the second beamcombiner, thereby combining the first image and the second image into astereoscopic virtual image; at the second beam combiner, passing animage of a user interaction volume through the second beam combinertoward the user view position and reflecting the stereoscopic virtualimage toward the view position.
 19. The method of claim 18, furthercomprising: tracking an eye position of the viewer and adjusting thefirst image and the second image to compensate for the eye position.